Switch from m2r to sphinx-mdinclude #127
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Merged
13 tasks
|
Just tested master + this PR and pytest is failing for me + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-sphinxcontrib-openapi-0.7.0-5.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-sphinxcontrib-openapi-0.7.0-5.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra --import-mode=importlib
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.15, pytest-7.1.3, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/openapi-0.7.0, configfile: tox.ini
collected 443 items
tests/test_openapi.py ................................. [ 7%]
tests/test_schema_utils.py ......... [ 9%]
tests/test_spec_examples.py ......... [ 11%]
tests/lib2to3/test_convert.py ............ [ 14%]
tests/lib2to3/test_convert_operation.py ...... [ 15%]
tests/lib2to3/test_convert_parameter.py ................ [ 19%]
tests/lib2to3/test_convert_parameters.py ... [ 20%]
tests/lib2to3/test_convert_path.py ........... [ 22%]
tests/lib2to3/test_convert_paths.py .... [ 23%]
tests/lib2to3/test_convert_request_body.py ..... [ 24%]
tests/lib2to3/test_convert_request_body_formdata.py ............ [ 27%]
tests/lib2to3/test_convert_response.py ............. [ 30%]
tests/lib2to3/test_convert_responses.py .... [ 31%]
tests/renderers/httpdomain/test_render.py ...FF.... [ 33%]
tests/renderers/httpdomain/test_render_json_schema_description.py .................................................................................................. [ 56%]
[ 56%]
tests/renderers/httpdomain/test_render_operation.py ............. [ 59%]
tests/renderers/httpdomain/test_render_parameter.py .......................................... [ 68%]
tests/renderers/httpdomain/test_render_parameters.py ............. [ 71%]
tests/renderers/httpdomain/test_render_paths.py ......... [ 73%]
tests/renderers/httpdomain/test_render_request_body.py .... [ 74%]
tests/renderers/httpdomain/test_render_request_body_example.py ......................... [ 80%]
tests/renderers/httpdomain/test_render_response.py .................................... [ 88%]
tests/renderers/httpdomain/test_render_response_example.py .............................. [ 95%]
tests/renderers/httpdomain/test_render_responses.py ........ [ 97%]
tests/renderers/httpdomain/test_render_restructuredtext_markup.py .......... [100%]
================================================================================= FAILURES =================================================================================
_______________________________________________________________________ test_render[v2.0/uber.json] ________________________________________________________________________
testrenderer = <sphinxcontrib.openapi.renderers._httpdomain.HttpdomainRenderer object at 0x7f07c6721850>
testspec = {'basePath': '/v1', 'definitions': {'Activities': {'properties': {'count': {'description': 'Total number of items avai...com', 'info': {'description': 'Move your app forward with the Uber API', 'title': 'Uber API', 'version': '1.0.0'}, ...}
rendered = PosixPath('/home/tkloczko/rpmbuild/BUILD/openapi-0.7.0/tests/renderers/httpdomain/rendered')
def test_render(testrenderer, testspec, rendered):
testspec_name, testspec = testspec
rendered_markup = "\n".join(testrenderer.render_restructuredtext_markup(testspec))
> assert rendered_markup == rendered.joinpath(testspec_name + ".rst").read_text()
E AssertionError: assert '.. http:get:...ected error\n' == '.. http:get:...ected error\n'
E Skipping 1312 identical leading characters in diff, use -v to show
E tes**
E
E - .. role:: raw-html-md(raw)
E - :format: html
E -
E - ...
E
E ...Full output truncated (150 lines hidden), use '-vv' to show
tests/renderers/httpdomain/test_render.py:25: AssertionError
_______________________________________________________________________ test_render[v2.0/uber.yaml] ________________________________________________________________________
testrenderer = <sphinxcontrib.openapi.renderers._httpdomain.HttpdomainRenderer object at 0x7f07c663cd90>
testspec = {'basePath': '/v1', 'definitions': {'Activities': {'properties': {'count': {'description': 'Total number of items avai...com', 'info': {'description': 'Move your app forward with the Uber API', 'title': 'Uber API', 'version': '1.0.0'}, ...}
rendered = PosixPath('/home/tkloczko/rpmbuild/BUILD/openapi-0.7.0/tests/renderers/httpdomain/rendered')
def test_render(testrenderer, testspec, rendered):
testspec_name, testspec = testspec
rendered_markup = "\n".join(testrenderer.render_restructuredtext_markup(testspec))
> assert rendered_markup == rendered.joinpath(testspec_name + ".rst").read_text()
E AssertionError: assert '.. http:get:...ected error\n' == '.. http:get:...ected error\n'
E Skipping 1313 identical leading characters in diff, use -v to show
E tes**
E
E - .. role:: raw-html-md(raw)
E - :format: html
E -
E - ...
E
E ...Full output truncated (150 lines hidden), use '-vv' to show
tests/renderers/httpdomain/test_render.py:25: AssertionError
============================================================================= warnings summary =============================================================================
../../../../../usr/lib/python3.8/site-packages/pkg_resources/__init__.py:123
/usr/lib/python3.8/site-packages/pkg_resources/__init__.py:123: PkgResourcesDeprecationWarning: 1.17.1-unknown is an invalid version and will not be supported in a future release
warnings.warn(
../../../../../usr/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/version.py:111
/usr/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/version.py:111: DeprecationWarning: Creating a LegacyVersion has been deprecated and will be removed in the next major release
warnings.warn(
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========================================================================= short test summary info ==========================================================================
FAILED tests/renderers/httpdomain/test_render.py::test_render[v2.0/uber.json] - AssertionError: assert '.. http:get:...ected error\n' == '.. http:get:...ected error\n'
FAILED tests/renderers/httpdomain/test_render.py::test_render[v2.0/uber.yaml] - AssertionError: assert '.. http:get:...ected error\n' == '.. http:get:...ected error\n'
================================================================ 2 failed, 432 passed, 2 warnings in 7.10s =================================================================
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:79: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-98d24dec-bba3-4d76-baf7-40ffe34b971e/test_rmtree_errorhandler_rerai0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_rmtree_errorhandler_rerai0'
warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:79: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-98d24dec-bba3-4d76-baf7-40ffe34b971e/test_rmtree_errorhandler_reado0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_rmtree_errorhandler_reado0'
warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:79: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-98d24dec-bba3-4d76-baf7-40ffe34b971e/test_safe_delete_no_perms0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_safe_delete_no_perms0'
warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:79: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-98d24dec-bba3-4d76-baf7-40ffe34b971e/test_safe_set_no_perms0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_safe_set_no_perms0'
warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:79: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-98d24dec-bba3-4d76-baf7-40ffe34b971e/test_safe_get_no_perms0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_safe_get_no_perms0'
warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:79: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-98d24dec-bba3-4d76-baf7-40ffe34b971e
<class 'OSError'>: [Errno 39] Directory not empty: '/tmp/pytest-of-tkloczko/garbage-98d24dec-bba3-4d76-baf7-40ffe34b971e'
warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:79: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-dc31f309-b95a-4271-9667-1339f46c53a6/test_rmtree_errorhandler_rerai0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_rmtree_errorhandler_rerai0'
warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:79: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-dc31f309-b95a-4271-9667-1339f46c53a6/test_rmtree_errorhandler_reado0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_rmtree_errorhandler_reado0'
warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:79: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-dc31f309-b95a-4271-9667-1339f46c53a6/test_safe_delete_no_perms0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_safe_delete_no_perms0'
warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:79: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-dc31f309-b95a-4271-9667-1339f46c53a6/test_safe_set_no_perms0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_safe_set_no_perms0'
warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:79: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-dc31f309-b95a-4271-9667-1339f46c53a6/test_safe_get_no_perms0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_safe_get_no_perms0'
warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:79: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-dc31f309-b95a-4271-9667-1339f46c53a6
<class 'OSError'>: [Errno 39] Directory not empty: '/tmp/pytest-of-tkloczko/garbage-dc31f309-b95a-4271-9667-1339f46c53a6'
warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:79: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-41642e46-df79-48ac-ae9b-7dcabd12c5d1/test_rmtree_errorhandler_rerai0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_rmtree_errorhandler_rerai0'
warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:79: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-41642e46-df79-48ac-ae9b-7dcabd12c5d1/test_rmtree_errorhandler_reado0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_rmtree_errorhandler_reado0'
warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:79: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-41642e46-df79-48ac-ae9b-7dcabd12c5d1/test_safe_delete_no_perms0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_safe_delete_no_perms0'
warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:79: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-41642e46-df79-48ac-ae9b-7dcabd12c5d1/test_safe_set_no_perms0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_safe_set_no_perms0'
warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:79: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-41642e46-df79-48ac-ae9b-7dcabd12c5d1/test_safe_get_no_perms0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_safe_get_no_perms0'
warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:79: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-41642e46-df79-48ac-ae9b-7dcabd12c5d1
<class 'OSError'>: [Errno 39] Directory not empty: '/tmp/pytest-of-tkloczko/garbage-41642e46-df79-48ac-ae9b-7dcabd12c5d1'
warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:79: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-a2628007-f0c4-42c8-9f20-1e2e295151c4/test_safe_delete_no_perms0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_safe_delete_no_perms0'
warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:79: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-a2628007-f0c4-42c8-9f20-1e2e295151c4/test_safe_set_no_perms0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_safe_set_no_perms0'
warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:79: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-a2628007-f0c4-42c8-9f20-1e2e295151c4/test_safe_get_no_perms0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_safe_get_no_perms0'
warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:79: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-a2628007-f0c4-42c8-9f20-1e2e295151c4
<class 'OSError'>: [Errno 39] Directory not empty: '/tmp/pytest-of-tkloczko/garbage-a2628007-f0c4-42c8-9f20-1e2e295151c4'
warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:79: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-a718da0f-bb8c-48f5-bc67-8d453447c5ea/test_rmtree_errorhandler_rerai0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_rmtree_errorhandler_rerai0'
warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:79: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-a718da0f-bb8c-48f5-bc67-8d453447c5ea/test_rmtree_errorhandler_reado0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_rmtree_errorhandler_reado0'
warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:79: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-a718da0f-bb8c-48f5-bc67-8d453447c5ea/test_safe_delete_no_perms0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_safe_delete_no_perms0'
warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:79: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-a718da0f-bb8c-48f5-bc67-8d453447c5ea/test_safe_set_no_perms0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_safe_set_no_perms0'
warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:79: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-a718da0f-bb8c-48f5-bc67-8d453447c5ea/test_safe_get_no_perms0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_safe_get_no_perms0'
warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:79: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-a718da0f-bb8c-48f5-bc67-8d453447c5ea
<class 'OSError'>: [Errno 39] Directory not empty: '/tmp/pytest-of-tkloczko/garbage-a718da0f-bb8c-48f5-bc67-8d453447c5ea'
warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:79: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-2d3a5dfb-e58b-4cbe-a863-0126e0f50568/test_rmtree_errorhandler_rerai0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_rmtree_errorhandler_rerai0'
warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:79: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-2d3a5dfb-e58b-4cbe-a863-0126e0f50568/test_rmtree_errorhandler_reado0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_rmtree_errorhandler_reado0'
warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:79: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-2d3a5dfb-e58b-4cbe-a863-0126e0f50568/test_safe_delete_no_perms0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_safe_delete_no_perms0'
warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:79: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-2d3a5dfb-e58b-4cbe-a863-0126e0f50568/test_safe_set_no_perms0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_safe_set_no_perms0'
warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:79: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-2d3a5dfb-e58b-4cbe-a863-0126e0f50568/test_safe_get_no_perms0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_safe_get_no_perms0'
warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:79: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-2d3a5dfb-e58b-4cbe-a863-0126e0f50568
<class 'OSError'>: [Errno 39] Directory not empty: '/tmp/pytest-of-tkloczko/garbage-2d3a5dfb-e58b-4cbe-a863-0126e0f50568'
warnings.warn( |
|
@kloczek is it passing for you on |
I have no idea because on my build infra I cannot send build request because broken build dependencies when |
|
Is there a viable fork of this project? It would be good to be able to get this functionality to work again. |
|
Should we post: https://github.com/sphinx-contrib/github-administration to see if there is a path to reviving this project. I think we could setup the CIs again and start moving it forward. There seems to be about 10 small PRs that would benefit from a bit of maintenance. |
@kloczek Right, I could reproduce, not sure how I missed it the first time. Thanks! I adjusted the tests accordingly. It's a bit dirty because I had to adjust the base JSON/YAML files used for the tests, but apart from updating mdinclude, I don't have a better proposal right now. See the last commit I pushed for details. |
|
@hmaarrfk Sorry, I'm not familiar with the github-administration repo. When you say "reviving this project", you mean |
|
OK just tested upodated PR and I was able to commit it, build it and pass cleanly pytest + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-sphinxcontrib-openapi-0.7.0-5.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-sphinxcontrib-openapi-0.7.0-5.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.15, pytest-7.2.0, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/openapi-0.7.0, configfile: tox.ini
plugins: xdist-3.0.2
collected 496 items
tests/test_openapi.py ................................. [ 6%]
tests/test_schema_utils.py ......... [ 8%]
tests/test_spec_examples.py .............................................................. [ 21%]
tests/lib2to3/test_convert.py ............ [ 23%]
tests/lib2to3/test_convert_operation.py ...... [ 25%]
tests/lib2to3/test_convert_parameter.py ................ [ 28%]
tests/lib2to3/test_convert_parameters.py ... [ 28%]
tests/lib2to3/test_convert_path.py ........... [ 31%]
tests/lib2to3/test_convert_paths.py .... [ 32%]
tests/lib2to3/test_convert_request_body.py ..... [ 33%]
tests/lib2to3/test_convert_request_body_formdata.py ............ [ 35%]
tests/lib2to3/test_convert_response.py ............. [ 38%]
tests/lib2to3/test_convert_responses.py .... [ 39%]
tests/renderers/httpdomain/test_render.py ......... [ 40%]
tests/renderers/httpdomain/test_render_json_schema_description.py .................................................................................................. [ 60%]
[ 60%]
tests/renderers/httpdomain/test_render_operation.py ............. [ 63%]
tests/renderers/httpdomain/test_render_parameter.py .......................................... [ 72%]
tests/renderers/httpdomain/test_render_parameters.py ............. [ 74%]
tests/renderers/httpdomain/test_render_paths.py ......... [ 76%]
tests/renderers/httpdomain/test_render_request_body.py .... [ 77%]
tests/renderers/httpdomain/test_render_request_body_example.py ......................... [ 82%]
tests/renderers/httpdomain/test_render_response.py .................................... [ 90%]
tests/renderers/httpdomain/test_render_response_example.py .............................. [ 96%]
tests/renderers/httpdomain/test_render_responses.py ........ [ 97%]
tests/renderers/httpdomain/test_render_restructuredtext_markup.py .......... [100%]
============================================================================= warnings summary =============================================================================
../../../../../usr/lib/python3.8/site-packages/pkg_resources/__init__.py:123
/usr/lib/python3.8/site-packages/pkg_resources/__init__.py:123: PkgResourcesDeprecationWarning: 1.17.1-unknown is an invalid version and will not be supported in a future release
warnings.warn(
../../../../../usr/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/version.py:111
/usr/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/version.py:111: DeprecationWarning: Creating a LegacyVersion has been deprecated and will be removed in the next major release
warnings.warn(
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
===================================================================== 487 passed, 2 warnings in 16.08s ===================================================================== |
|
As you can see, this is currently failing CI. Not your fault though: the deps in |
m2r is unmaintained [0]. It relies on mistune, but is not compatible with mistune versions >= 2.0 [1]. This makes it impossible to update mistune, even though there is a CVE in the old versions of the package [2]. Switch to sphinx-mdinclude instead. There is also m2r2, but it is not a suitable alternative because they simply force the use of an older mistune version [3]. [0] sphinx-contrib#123 [1] miyakogi/m2r#66 [2] GHSA-fw3v-x4f2-v673 [3] CrossNox/m2r2#43 Fixes: sphinx-contrib#123
Since commit 1e95ac6 ("Actualize information in setup.py"), the dependency on m2r, now replaced with mdinclude, is no longer part of the "extras_require", but of the "install_requires" in setup.py. So the dependency should always be satisfied, and we don't need to handle the case where the import fails.
Two kind of changes are required to adjust the tests after the move from m2r to mdinclude for parsing Markdown. - We need to slightly adjust the expected results for the uber.json/uber.yaml tests, because of minor differences in formatting between m2r and mdinclude (namely, spaces before/after the inserted role markers). - Alas, we also need to adjust the base JSON/YAML files themselves, because it appears that mdinclude (itself or through its mistune dependency, I'm not sure, see [0] and [1]) does not support simple HTML tags without slash (e.g. "<div>...</div>" works, "<br />" too, but not simply "<br>"). [0] https://github.com/lepture/mistune/blob/693db5c/mistune/inline_parser.py#L32 [1] https://github.com/amyreese/sphinx-mdinclude/blob/3636e33fe83cfcfea14e70fd503777ceb11f4401/sphinx_mdinclude/parse.py#L62
stephenfin
force-pushed
the
pr/mdinclude
branch
from
a65289f to
b8debb4
Compare
|
I've actually gone ahead and rebased this myself. Let's see how this passes. There was a small formatting issue on the first commit so I fixed that as part of the rebase also. |
|
It is yet another thing. In sphinxcontrib/init.py On python/importlib_metadata#378 is poasible to find some hints how to migrate to |
|
Like this? #137 |
|
Yes 😄 |
13 tasks
qmonnet
added a commit
to cilium/cilium
that referenced
this pull request
Jan 17, 2023
Once upon a time, Cilium docs used the openapi Sphinx add-on to generate its API reference based on the code. And things were good. One day, Dependabot raised a security alert, stating that Mistune v2.0.2 was vulnerable to catastrophic backtracking [0] - this is a regex parsing thing. Mistune was a dependency to m2r, an add-on to parse Markdown in Sphinx, which in turn was a dependency to openapi. The easy path would have been to update m2r to use the latest, fixed Mistune version; but m2r was incompatible with Mistune >= 2.0.0, and also it was no longer in development. There was a fork, m2r2, which had little activity, and would avoid the security issue by very simply pinning the Mistune version to 0.8.4 (which would either fail to build Cilium's reference correctly, or bring some incompatibilities with other dependencies, at this point the narrator does not remember for sure). There was a fork of the fork, sphinx-mdinclude. We could use that project to update openapi, except that it was not compatible with recent versions of docutils, and that this would cause openapi's test suite to fail to pass. ... So we ended up forking the openapi repository to update the dependency to sphinx-mdinclude locally, and this is what we've been using since last summer. And things were good again. But things are even better when they go upstream [citation needed]. We also filed the issue for docutils compatibility in sphinx-mdinclude [1]. It was fixed (thanks!). We submitted a PR to have openapi switch to sphinx-mdinclude [2]. It was adjusted (thanks!), merged, and a new tag was created. Now at last, we can switch back to the upstream version of openapi! [And the build system lived happily ever after.] [0]: GHSA-fw3v-x4f2-v673 [1]: omnilib/sphinx-mdinclude#8 [2]: sphinx-contrib/openapi#127 I did _not_ run `make -C Documentation update-requirements`, because the resulting changes seemed to break the Netlify preview [3]. I stuck to openapi and bumped sphinx-mdinclude to >= 0.5.2, as required by openapi. [3] https://app.netlify.com/sites/docs-cilium-io/deploys/63c55fcc5531c6000838b87c Signed-off-by: Quentin Monnet <[email protected]>
joestringer
pushed a commit
to cilium/cilium
that referenced
this pull request
Feb 4, 2023
Once upon a time, Cilium docs used the openapi Sphinx add-on to generate its API reference based on the code. And things were good. One day, Dependabot raised a security alert, stating that Mistune v2.0.2 was vulnerable to catastrophic backtracking [0] - this is a regex parsing thing. Mistune was a dependency to m2r, an add-on to parse Markdown in Sphinx, which in turn was a dependency to openapi. The easy path would have been to update m2r to use the latest, fixed Mistune version; but m2r was incompatible with Mistune >= 2.0.0, and also it was no longer in development. There was a fork, m2r2, which had little activity, and would avoid the security issue by very simply pinning the Mistune version to 0.8.4 (which would either fail to build Cilium's reference correctly, or bring some incompatibilities with other dependencies, at this point the narrator does not remember for sure). There was a fork of the fork, sphinx-mdinclude. We could use that project to update openapi, except that it was not compatible with recent versions of docutils, and that this would cause openapi's test suite to fail to pass. ... So we ended up forking the openapi repository to update the dependency to sphinx-mdinclude locally, and this is what we've been using since last summer. And things were good again. But things are even better when they go upstream [citation needed]. We also filed the issue for docutils compatibility in sphinx-mdinclude [1]. It was fixed (thanks!). We submitted a PR to have openapi switch to sphinx-mdinclude [2]. It was adjusted (thanks!), merged, and a new tag was created. Now at last, we can switch back to the upstream version of openapi! [And the build system lived happily ever after.] [0]: GHSA-fw3v-x4f2-v673 [1]: omnilib/sphinx-mdinclude#8 [2]: sphinx-contrib/openapi#127 I did _not_ run `make -C Documentation update-requirements`, because the resulting changes seemed to break the Netlify preview [3]. I stuck to openapi and bumped sphinx-mdinclude to >= 0.5.2, as required by openapi. [3] https://app.netlify.com/sites/docs-cilium-io/deploys/63c55fcc5531c6000838b87c Signed-off-by: Quentin Monnet <[email protected]>
aanm
pushed a commit
to pchaigno/cilium
that referenced
this pull request
Feb 13, 2023
[ upstream commit e4889d7 ] Once upon a time, Cilium docs used the openapi Sphinx add-on to generate its API reference based on the code. And things were good. One day, Dependabot raised a security alert, stating that Mistune v2.0.2 was vulnerable to catastrophic backtracking [0] - this is a regex parsing thing. Mistune was a dependency to m2r, an add-on to parse Markdown in Sphinx, which in turn was a dependency to openapi. The easy path would have been to update m2r to use the latest, fixed Mistune version; but m2r was incompatible with Mistune >= 2.0.0, and also it was no longer in development. There was a fork, m2r2, which had little activity, and would avoid the security issue by very simply pinning the Mistune version to 0.8.4 (which would either fail to build Cilium's reference correctly, or bring some incompatibilities with other dependencies, at this point the narrator does not remember for sure). There was a fork of the fork, sphinx-mdinclude. We could use that project to update openapi, except that it was not compatible with recent versions of docutils, and that this would cause openapi's test suite to fail to pass. ... So we ended up forking the openapi repository to update the dependency to sphinx-mdinclude locally, and this is what we've been using since last summer. And things were good again. But things are even better when they go upstream [citation needed]. We also filed the issue for docutils compatibility in sphinx-mdinclude [1]. It was fixed (thanks!). We submitted a PR to have openapi switch to sphinx-mdinclude [2]. It was adjusted (thanks!), merged, and a new tag was created. Now at last, we can switch back to the upstream version of openapi! [And the build system lived happily ever after.] [0]: GHSA-fw3v-x4f2-v673 [1]: omnilib/sphinx-mdinclude#8 [2]: sphinx-contrib/openapi#127 I did _not_ run `make -C Documentation update-requirements`, because the resulting changes seemed to break the Netlify preview [3]. I stuck to openapi and bumped sphinx-mdinclude to >= 0.5.2, as required by openapi. [3] https://app.netlify.com/sites/docs-cilium-io/deploys/63c55fcc5531c6000838b87c Signed-off-by: Quentin Monnet <[email protected]> Signed-off-by: Paul Chaignon <[email protected]>
aanm
pushed a commit
to cilium/cilium
that referenced
this pull request
Feb 13, 2023
[ upstream commit e4889d7 ] Once upon a time, Cilium docs used the openapi Sphinx add-on to generate its API reference based on the code. And things were good. One day, Dependabot raised a security alert, stating that Mistune v2.0.2 was vulnerable to catastrophic backtracking [0] - this is a regex parsing thing. Mistune was a dependency to m2r, an add-on to parse Markdown in Sphinx, which in turn was a dependency to openapi. The easy path would have been to update m2r to use the latest, fixed Mistune version; but m2r was incompatible with Mistune >= 2.0.0, and also it was no longer in development. There was a fork, m2r2, which had little activity, and would avoid the security issue by very simply pinning the Mistune version to 0.8.4 (which would either fail to build Cilium's reference correctly, or bring some incompatibilities with other dependencies, at this point the narrator does not remember for sure). There was a fork of the fork, sphinx-mdinclude. We could use that project to update openapi, except that it was not compatible with recent versions of docutils, and that this would cause openapi's test suite to fail to pass. ... So we ended up forking the openapi repository to update the dependency to sphinx-mdinclude locally, and this is what we've been using since last summer. And things were good again. But things are even better when they go upstream [citation needed]. We also filed the issue for docutils compatibility in sphinx-mdinclude [1]. It was fixed (thanks!). We submitted a PR to have openapi switch to sphinx-mdinclude [2]. It was adjusted (thanks!), merged, and a new tag was created. Now at last, we can switch back to the upstream version of openapi! [And the build system lived happily ever after.] [0]: GHSA-fw3v-x4f2-v673 [1]: omnilib/sphinx-mdinclude#8 [2]: sphinx-contrib/openapi#127 I did _not_ run `make -C Documentation update-requirements`, because the resulting changes seemed to break the Netlify preview [3]. I stuck to openapi and bumped sphinx-mdinclude to >= 0.5.2, as required by openapi. [3] https://app.netlify.com/sites/docs-cilium-io/deploys/63c55fcc5531c6000838b87c Signed-off-by: Quentin Monnet <[email protected]> Signed-off-by: Paul Chaignon <[email protected]>
pchaigno
pushed a commit
to pchaigno/cilium
that referenced
this pull request
Feb 15, 2023
[ upstream commit e4889d7 ] Once upon a time, Cilium docs used the openapi Sphinx add-on to generate its API reference based on the code. And things were good. One day, Dependabot raised a security alert, stating that Mistune v2.0.2 was vulnerable to catastrophic backtracking [0] - this is a regex parsing thing. Mistune was a dependency to m2r, an add-on to parse Markdown in Sphinx, which in turn was a dependency to openapi. The easy path would have been to update m2r to use the latest, fixed Mistune version; but m2r was incompatible with Mistune >= 2.0.0, and also it was no longer in development. There was a fork, m2r2, which had little activity, and would avoid the security issue by very simply pinning the Mistune version to 0.8.4 (which would either fail to build Cilium's reference correctly, or bring some incompatibilities with other dependencies, at this point the narrator does not remember for sure). There was a fork of the fork, sphinx-mdinclude. We could use that project to update openapi, except that it was not compatible with recent versions of docutils, and that this would cause openapi's test suite to fail to pass. ... So we ended up forking the openapi repository to update the dependency to sphinx-mdinclude locally, and this is what we've been using since last summer. And things were good again. But things are even better when they go upstream [citation needed]. We also filed the issue for docutils compatibility in sphinx-mdinclude [1]. It was fixed (thanks!). We submitted a PR to have openapi switch to sphinx-mdinclude [2]. It was adjusted (thanks!), merged, and a new tag was created. Now at last, we can switch back to the upstream version of openapi! [And the build system lived happily ever after.] [0]: GHSA-fw3v-x4f2-v673 [1]: omnilib/sphinx-mdinclude#8 [2]: sphinx-contrib/openapi#127 I did _not_ run `make -C Documentation update-requirements`, because the resulting changes seemed to break the Netlify preview [3]. I stuck to openapi and bumped sphinx-mdinclude to >= 0.5.2, as required by openapi. [3] https://app.netlify.com/sites/docs-cilium-io/deploys/63c55fcc5531c6000838b87c Signed-off-by: Quentin Monnet <[email protected]> Signed-off-by: Paul Chaignon <[email protected]>
pchaigno
pushed a commit
to cilium/cilium
that referenced
this pull request
Feb 16, 2023
[ upstream commit e4889d7 ] Once upon a time, Cilium docs used the openapi Sphinx add-on to generate its API reference based on the code. And things were good. One day, Dependabot raised a security alert, stating that Mistune v2.0.2 was vulnerable to catastrophic backtracking [0] - this is a regex parsing thing. Mistune was a dependency to m2r, an add-on to parse Markdown in Sphinx, which in turn was a dependency to openapi. The easy path would have been to update m2r to use the latest, fixed Mistune version; but m2r was incompatible with Mistune >= 2.0.0, and also it was no longer in development. There was a fork, m2r2, which had little activity, and would avoid the security issue by very simply pinning the Mistune version to 0.8.4 (which would either fail to build Cilium's reference correctly, or bring some incompatibilities with other dependencies, at this point the narrator does not remember for sure). There was a fork of the fork, sphinx-mdinclude. We could use that project to update openapi, except that it was not compatible with recent versions of docutils, and that this would cause openapi's test suite to fail to pass. ... So we ended up forking the openapi repository to update the dependency to sphinx-mdinclude locally, and this is what we've been using since last summer. And things were good again. But things are even better when they go upstream [citation needed]. We also filed the issue for docutils compatibility in sphinx-mdinclude [1]. It was fixed (thanks!). We submitted a PR to have openapi switch to sphinx-mdinclude [2]. It was adjusted (thanks!), merged, and a new tag was created. Now at last, we can switch back to the upstream version of openapi! [And the build system lived happily ever after.] [0]: GHSA-fw3v-x4f2-v673 [1]: omnilib/sphinx-mdinclude#8 [2]: sphinx-contrib/openapi#127 I did _not_ run `make -C Documentation update-requirements`, because the resulting changes seemed to break the Netlify preview [3]. I stuck to openapi and bumped sphinx-mdinclude to >= 0.5.2, as required by openapi. [3] https://app.netlify.com/sites/docs-cilium-io/deploys/63c55fcc5531c6000838b87c Signed-off-by: Quentin Monnet <[email protected]> Signed-off-by: Paul Chaignon <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
m2r is unmaintained. It relies on mistune, but is not compatible with mistune versions >= 2.0. This makes it impossible to update mistune, even though there is a CVE in the old versions of the package.
Switch to sphinx-mdinclude instead.
Note: There is also m2r2, but it is not look like a suitable alternative because the project simply forces the use of an older mistune version.
I haven't managed to pass the full CI locally successfully, but the issues seemed to be related to my setup and not to the changes from this PR. If they did come from the PR, I would welcome some assistance to address them.
Fixes: #123
Possibly addresses #121 as well